Multimodal Affective States Recognition Based on Multiscale CNNs and Biologically Inspired Decision Fusion Model

There has been an encouraging progress in the affective states recognition models based on single-modality signals as electroencephalogram (EEG) or peripheral physiological recent years. However, multimodal signals-based methods have not thoroughly exploited yet. Here we propose Multiscale Convolutional Neural Networks (Multiscale CNNs) and a biologically inspired decision fusion model for recognition. Firstly, raw are pre-processed with baseline signals. Then, High Scale CNN Low CNNs utilized to predict probability of output EEG each signal respectively. Finally, calculates reliability by Euclidean distance between various class labels classification from CNNs, is made more reliable modality information while other modalities retained. We use this classify four arousal valence plane DEAP AMIGOS dataset. The results show that improves accuracy significantly compared result signals, achieve 98.52% 99.89% dataset